Skip to content

Fix/amd wheel jit kernel support#22661

Open
akao-amd wants to merge 3 commits intosgl-project:mainfrom
akao-amd:fix/amd-wheel-jit-kernel-support
Open

Fix/amd wheel jit kernel support#22661
akao-amd wants to merge 3 commits intosgl-project:mainfrom
akao-amd:fix/amd-wheel-jit-kernel-support

Conversation

@akao-amd
Copy link
Copy Markdown
Contributor

@akao-amd akao-amd commented Apr 13, 2026

Motivation

Fix #19917 .

Modifications

  • Include a necessary dependency: apache-tvm-ffi.
  • Miscellaneous clean-ups.

Accuracy Tests

$ pip install amd-sglang[all-hip,rocm720]==0.5.10rc0 --pre -i https://pypi.amd.com/sglang/simple --extra-index-url https://pypi.org/simple
$ cd /opt/venv/lib/python3.10/site-packages/sglang/jit_kernel/tests
$ pytest diffusion/test_qwen_image_modulation.py
================================================ test session starts =================================================
platform linux -- Python 3.10.12, pytest-9.0.2, pluggy-1.6.0
rootdir: /opt/venv/lib/python3.10/site-packages/sglang/jit_kernel/tests
plugins: hypothesis-6.150.2, anyio-4.13.0
collected 288 items

diffusion/test_qwen_image_modulation.py ...................................................................... [ 24%]
.............................................................................................................. [ 62%]
............................................................................................................   [100%]

================================================== warnings summary ==================================================
<frozen importlib._bootstrap>:241
  <frozen importlib._bootstrap>:241: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:241
  <frozen importlib._bootstrap>:241: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================== 288 passed, 2 warnings in 13.99s ==========================================
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute

Speed Tests and Profiling

N/A.

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

akao-amd and others added 3 commits April 13, 2026 10:06
@akao-amd @claude
[AMD] Enable JIT kernel support for wheel installations
82ff9e4 
This commit enables JIT (Just-In-Time) kernel compilation for users
installing the amd-sglang wheel, addressing issue sgl-project#19917.

Changes:
1. Add apache-tvm-ffi dependency to AMD wheel runtime_common
   - JIT kernel compilation requires apache-tvm-ffi for FFI integration
   - This dependency was present in the main pyproject.toml but missing
     from the AMD wheel's runtime_common

2. Document JIT kernel support in AMD wheel README
   - Add section explaining JIT kernel requirements
   - Include verification steps for users
   - Note system requirements (compiler toolchain)

The JIT kernel source files were already being packaged via the
`jit_kernel/**/*` pattern in package-data. The _resolve_kernel_path()
function in jit_kernel/utils.py already handles both editable and wheel
installs correctly since source files are included in package data.
The main issue was the missing apache-tvm-ffi dependency.

Fixes sgl-project#19917

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@akao-amd @claude
[AMD] Sync AMD wheel dependencies with main pyproject_other.toml
63daa07 
Synchronize 3rdparty/amd/wheel/sglang/pyproject.toml with recent changes
from python/pyproject_other.toml to ensure AMD wheel users have access
to the same features and bug fixes as other platforms.

Key updates:
1. Dependencies updated:
   - transformers: 4.57.1 → 5.3.0
   - Added mistral_common>=1.9.0 (required for transformers 5.x)
   - sglang-kernel: 0.4.0 → 0.4.1 (both rocm700 and rocm720)
   - diffusers: upgraded to 0.37.0
   - cache-dit: 1.1.8 → 1.3.0
   - runai_model_streamer: 0.15.5 → 0.15.7
   - peft: unversioned → >=0.18.0
   - Added av (video processing)

2. Removed AMD-specific extras that were in main:
   - hf_transfer, huggingface_hub, decord2 (not in main's runtime_common)

3. Refactored diffusion dependencies:
   - Extracted diffusion_common to reduce duplication
   - Both diffusion_hip and diffusion_musa now reference diffusion_common
   - Follows the same pattern as python/pyproject_other.toml

4. Test dependencies updated:
   - Added: polars, pytest-cov, diff-cover, bitsandbytes, parameterized
   - Updated peft to >=0.18.0
   - Removed gguf (not in main's test deps)

5. Added killall_sglang script entry point

6. Added tracing to all_hip extras

These changes ensure AMD wheel installations have feature parity with
other platforms and benefit from recent dependency updates and bug fixes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@akao-amd
[AMD] Remove MUSA and HPU sections from AMD wheel config
75ff225 
These platform-specific sections are not relevant for AMD wheels.
@gemini-code-assist
Copy link
Copy Markdown
Contributor

gemini-code-assist bot commented Apr 13, 2026

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@github-actions github-actions bot added documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file labels Apr 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[AMD] Support JIT kernel for wheels

1 participant

@akao-amd